As an information-oriented society has been come, person identification techniques for discriminating a person from others have become more important, and, thus there have been significant number of studies in the field of personal information protection and person identification through a computer using biometrical technologies. In biometrical technologies, facial recognition technique becomes the most convenient and competitive technique since it does not require a specified action or behavior of a user and employs a non-contact manner. The facial recognition technique is widely used in various applications such as identification, human-computer interface (HCI), and access control. However, there are several drawbacks in the facial recognition technique. One of these drawbacks is deformation of facial images occurred by glasses.
To remove glasses from a facial image with the glasses, various image processing methods are proposed: an algorithm for extracting glasses from a facial image using a deformable contour to remove the extracted glasses; an algorithm for removing small occlusion regions such as certain facial regions occluded by glasses using a flexible model that is called as an active appearance model; and an image processing method using PCA algorithm.
An image processing method using PCA algorithm is now widely used. The PCA algorithm is classified into two processes. One is a training process for extracting eigenfaces from a plurality of unspecified sample glassless facial images Γ N, wherein N=1, 2, . . . ,M. The sample facial images ΓN include facial images of an individual and/or another individuals. The other is a process for obtaining glassless reconstruction images from current input facial images Γ with glasses by using the extracted eigenfaces.
Descriptions of the training process for extracting eigenfaces will be first described in detail. An average image φ are calculated from the sample facial images ΓN for use in the training process by using Equation 1 and the average image φ is subtracted from the sample facial images ΓN as expressed in Equation 2, wherein each of the sample facial images ΓN is expressed as a column vector.
                    φ        =                              1            M                    ⁢                                    ∑                              N                =                1                            M                        ⁢                                          Γ                N                            .                                                          (                  Eq          .                                          ⁢          1                )                                          Φ          N                =                              Γ            N                    -                      φ            .                                              (                  Eq          .                                          ⁢          2                )            
Then, a covariance matrix C with respect to the sample facial images ΓN is obtained from differential images ΦN, which is calculated by subtracting the average image φ from each of the sample facial images ΓN by using the following Equation 3.
                              C          =                                                    1                M                            ⁢                                                ∑                                      N                    =                    1                                    M                                ⁢                                                      Φ                    N                                    ⁢                                      Φ                    N                    T                                                                        =                          AA              T                                      ⁢                                  ⁢                  A          =                      [                                          Φ                1                            ,                              Φ                2                            ,              ⋯              ⁢                                                          ,                              Φ                M                                      ]                                              (                  Eq          .                                          ⁢          3                )            wherein A is a matrix composed of the differential images ΦN and AT is a transpose of A.
Consequently, eigenvectors are obtained from the covariance matrix C, wherein the eigenvectors is referred to eigenfaces uk(k=1, . . . ,M). Detailed description for a process of obtaining the eigenfaces uk will be omitted because this process is well known to those skilled in the art.
Next, the input facial images Γ with glasses are expressed as glassless reconstruction images {circumflex over (Γ)} by using the eigenfaces uk. With the following Equation 4, the average image φ is subtracted from the input facial images Γ, and the resultant is projected to the respective eigenfaces uk.ωk=ukT(Γ−φ), k=1, . . . ,M  (Eq. 4)wherein ωk is a weight which allows the input facial images Γ to be expressed on a space consisting of the eigenfaces uk. The reconstruction images {circumflex over (Γ)} are also expressed in terms of the sum of weights of the eigenfaces uk extracted from the sample facial images ΓN by using the following Equation 5.
                                          Γ            ^                    =                      φ            +                                          ∑                                  k                  =                  1                                                  M                  ′                                            ⁢                                                ω                  k                                ⁢                                  u                  k                                                                    ,                                  ⁢                              M            ′                    ≤          M                                    (                  Eq          .                                          ⁢          5                )            wherein a number of the eigenfaces uk required is equal to M or to M′ less than M, M being a total number of the eigenfaces uk.
Where eigenfaces uk are extracted from the sample facial images ΓN as described above, the extracted eigenfaces uk include facial characteristics only so that final glassless facial images can be obtained by reconstructing the input facial images Γ on the basis of the extracted eigenfaces uk to produce the reconstruction images {circumflex over (Γ)}. However, the reconstruction images {circumflex over (Γ)} produced according to the conventional method have many errors thereon. Referring to FIG. 1, which shows that glasses are not removed completely although the reconstruction images {circumflex over (Γ)} are similar to the input facial images Γ, and there are numerous errors over the reconstruction images {circumflex over (Γ)}. In FIG. 1, “client” is a person included in a training set and “non-client” is a person excluded in the training set. Although there are numerous errors as shown in FIG. 1, the reconstruction images {circumflex over (Γ)} of “clients” are better than those of “non-clients” in quality since facial characteristics are reflected in the extracted eigenfaces uk.
However, there are some problems in regarding the reconstruction images {circumflex over (Γ)} obtained according to the conventional method as complete glassless facial images. Firstly, if the reconstruction images {circumflex over (Γ)} are generated with respect to the input facial images Γ on the basis of the eigenfaces uk that are extracted from the sample facial images ΓN included in the training set, particular characteristics of the input facial images Γ would not be appeared on the reconstruction images {circumflex over (Γ)}. Secondly, if occlusion regions due to glasses are considerable in the input facial images Γ, the reconstruction images {circumflex over (Γ)} will include many errors thereon so that these may appear to be unnatural and different from the input facial images Γ.
As described above, since problems due to glasses in the input facial images Γ are merely regarded as the matter of glasses frame, many limitations are arisen in the conventional methods so that obtaining high quality glassless facial images is very difficult.